New l1-Norm Relaxations and Optimizations for Graph Clustering

In recent data mining research, the graph clustering methods, such as normalized cut and ratio cut, have been well studied and applied to solve many unsupervised learning applications. The original graph clustering methods are NP-hard problems. Traditional approaches used spectral relaxation to solve the graph clustering problems. The main disadvantage of these approaches is that the obtained spectral solutions could severely deviate from the true solution. To solve this problem, in this paper, we propose a new relaxation mechanism for graph clustering methods. Instead of minimizing the squared distances of clustering results, we use the 1-norm distance. More important, considering the normalized consistency, we also use the 1norm for the normalized terms in the new graph clustering relaxations. Due to the sparse result from the 1-norm minimization, the solutions of our new relaxed graph clustering methods get discrete values with many zeros, which are close to the ideal solutions. Our new objectives are difficult to be optimized, because the minimization problem involves the ratio of nonsmooth terms. The existing sparse learning optimization algorithms cannot be applied to solve this problem. In this paper, we propose a new optimization algorithm to solve this difficult non-smooth ratio minimization problem. The extensive experiments have been performed on three two-way clustering and eight multi-way clustering benchmark data sets. All empirical results show that our new relaxation methods consistently enhance the normalized cut and ratio cut clustering results. Introduction Clustering is an important task in computer vision and machine learning research with many applications, such as image segmentation (Shi and Malik 2000), image categorization (Grauman and Darrell 2006), scene analysis (Koppal and Narasimhan 2006), motion modeling (P.Ochs and T.Brox 2012), and medical image analysis (Brun, Park, and ∗To whom all correspondence should be addressed. This work was partially supported by the following grants: NSF-IIS 1117965, NSF-IIS 1302675, NSF-IIS 1344152, NSF-DBI 1356628, NSF-IIS 1423591, NIH R01 AG049371. Copyright c © 2016, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Shenton 2004). In the past decades, many clustering algorithms have been proposed. Among these approaches, the use of manifold information in graph clustering has shown the state-of-the-art clustering performance. The graph based clustering methods model the data as a weighted undirected graph based on the pair-wise similarities. Clustering is then accomplished by finding the best cuts of the graph that optimize the predefined cost functions. Two types of graph clustering methods, normalized cut (Shi and Malik 2000) and ratio cut (Cheng and Wei 1991; Hagen and Kahng 1992), are popularly used to solve the clustering problems due to their good clustering performance. Solving the graph clustering problem is a difficult task (NP-hard problems). The main difficulty of the graph clustering problem comes from the constrains on the solution. It is hard to solve the graph clustering problems exactly. However, the approximation solutions are possible with spectral relaxations. The optimization usually leads to the computation of the top eigenvectors of certain graph affinity matrices, and the clustering result can be derived from the obtained eigen-space. However, the traditional spectral relaxations lead the non-optimal clustering results. The spectral solutions don’t directly provide the clustering results and the thresholding post-processing has to be applied, such that the results often severely deviate from the true solution. More recently, tight relaxations of balanced graph clustering methods were proposed (Bühler and Hein 2009; Luo et al. 2010; Hein and Setzer 2011), and gradient based method was used to solve the problem, which is time consuming and slow to converge in practice. In order to solve the above challenging issues, in this paper, we revisit the normalized cut and ratio cut methods, and propose new relaxations for these methods to achieve the discrete and sparse clustering results which are close to the ideal solutions. Instead of minimizing the projected squared clustering indictors distance, we minimize the 1 distance. Meanwhile, our new relaxations also use the 1-norm for the normalization terms. Due to the 1-normminimization, most elements of each clustering indictor are enforced to be zero and hence the clustering results are close the ideal solutions. However, our new relations introduce a difficult optimization problem which optimizes the ratio of two nonsmooth terms. The standard optimization methods for sparse learning, such as Proximal Gradient, Iterative ShrinkageProceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16)